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I LETTER I 

A Phenomenological Study on Threshold Improvement via Spatial 
Coupling* 

Keigo TAKEUCHI 1 a) , Toshiyuki TANAKA , , b) , and Tsutomu KAWABATA 1 c) , Members 



SUMMARY Kudekar et al. proved an interesting result in low-density 
parity-check (LDPC) convolutional codes: The belief-propagation (BP) 
threshold is boosted to the maximum-a-posteriori (MAP) threshold by spa- 
tial coupling. Furthermore, the authors showed that the BP threshold for 
code-division multiple-access (CDMA) systems is improved up to the op- 
timal one via spatial coupling. In this letter, a phenomenological model for 
elucidating the essence of these phenomenon, called threshold improve- 
ment, is proposed. The main result implies that threshold improvement 
occurs for spatially-coupled general graphical models. 
keywords: spatial coupling, threshold saturation, belief propagation (BP), 
dynamical systems. 

1. Introduction 

Low-density parity-check (LDPC) convolutional codes have 
been shown to outperform conventional LDPC block codes 
when iterative decoders based on belief propagation (BP) 
are used 0][2). An LDPC convolutional code is constructed 
as a one-dimensionally coupled chain of LDPC block codes. 
As an explanation of this interesting result, it has been 
shown theoretically [3 4| and numerically [5| that the BP 
threshold of an LDPC convolutional code is boosted to 
the maximum-a-posteriori (MAP) one of the corresponding 
LDPC block code. This phenomenon is called "threshold 
saturation" via spatial coupling ]3]|4). 

Recently, we showed that the threshold saturation also 
occurs in spatially-coupled code-division multiple-access 
(CDMA) systems: The BP threshold for sparsely-spread 
CDMA systems is boosted to the optimal one via spatial 
coupling ||6][7). Since it is unclear whether the BP thresh- 
old for any graphical model can be improved to the optimal 
one, the term "threshold improvement" via spatial coupling 
is used in this letter, instead of threshold saturation. It is be- 
lieved that threshold improvement is universal, i.e., the per- 
formance of the BP algorithm can be improved by coupling 
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general graphical models spatially. This conjecture has been 
verified for several graphical models ||8]|9]. The purpose of 
this letter is to present a phenomenological study that sup- 
ports the universality of threshold improvement. 

Threshold improvement is a static property of spatially- 
coupled graphical models, rather than dynamical properties. 
It is well-known that BP can calculate the MAP solution 
exactly if the factor graph defining the BP is a tree ifTUI . 
Furthermore, if the BP algorithm for a general factor graph 
converges, the BP fixed-points correspond to the station- 
ary points of the so-called Bethe free energy for the fac- 
tor graph [11 1, while the MAP solution corresponds to the 
global minimizer of the true free energy. Roughly speaking, 
the Bethe free energy is obtained by locally approximating 
the original factor graph by trees. These results allow us 
to characterize the static properties of the BP algorithm by 
stationary solutions to a dynamical system that has a poten- 
tial energy function whose fixed-points coincide with those 
for the Bethe free energy, which is analogous to the density 
evolution (DE) equation for LDPC codes. In this letter, we 
restrict graphical models to such a class of graphical models 
that BP algorithms converge asymptotically. 

As a phenomenological model for elucidating thresh- 
old improvement, we propose a spatially-coupled dynami- 
cal system with multiple stable solutions. This letter is or- 
ganized as follows: In Section [2] the BP algorithm is char- 
acterized via a dynamical system with multiple stable so- 
lutions after presenting a motivating example for regular 
LDPC codes. In Section [3] a spatially-coupled dynamical 
system with multiple stable solutions is defined. Further- 
more, an intuitive understanding of threshold improvement 
is presented. Section|4]presents the main result of this letter. 

2. Systems without Spatial Coupling 

2. 1 Density Evolution for regular LDPC Codes 

The explicit formula of the Bethe free energy for LDPC 
codes is unknown. We shall construct potential energy as- 
sociated with the Bethe free energy from DE. Let us con- 
sider the DE equation for regular LDPC codes over binary 
erasure channel (BEC) with erasure probability e lfT2l The- 
orem 3.50] 

dU 

y r +i -y t = —r-iyu «0> yo = l. (l) 
dy 

where the potential energy associated with the Bethe free 
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energy is given by 

U(y;e)= f {z - eA(c(z))} dz, 



U(y) 



(2) 



with c(y) = 1 -p(l - y). In these expressions, y, denotes 
the bit error rate (BER) for the BP decoder in iteration t. 
Furthermore, A(y) and p(y) are given by A{y) = if" and 
p(y) - y^ 1 ! respectively, with d v and d c denoting the de- 
grees of variable and check nodes. Note that the BP fixed- 
point i/oo corresponds to a stationary point of the potential 
energy ©. When e is smaller than the BP threshold e^p, the 
potential (f2]i has the unique stable solution y\ = 0. This ob- 
servation implies that the BER converges to zero in t — > oo 
for e < £bp- When e > eBP, on the other hand, the po- 
tential © has two stable solutions y\ — and y r > 0, and 
one unstable solution y u > 0, satisfying y\ < y u < y r . The 
BER in this case converges to the strictly positive value y r 
in t — > oo. Threshold saturation [4| implies that y, can ap- 
proach the smaller stable solution y\ — for e e [£bp, £map) 
by spatial coupling, with 6map denoting the MAP threshold. 

2.2 Dynamical System with Multiple Stable Solutions 

In order to investigate the universality of threshold improve- 
ment, we consider a continuous-time dynamical system with 
a general potential energy function U (y) 

dy dU 



dt 



dy 



(y), 2/(0) = 2/0 6 • 



(3) 



As a simple example, let us consider a bistable potential en- 
ergy function U(y), shown in Fig.Q] The potential energy 
U{y) has two stable solutions and one unstable solution y u . 
Let j/_ and y + denote the smaller and larger stable solutions, 
respectively, i.e., (/_ < y + . If the initial value yo is larger than 
the unstable solution y u , the state y(t) converges to the larger 
stable solution y + in t — > oo. Otherwise, y(f) — > Without 
loss of generality, we assume that larger y{t) implies better 
performance. The larger stable solution y+ corresponds to 
the optimal solution. Note that the term "optimal solution" 
is used to represent not the global stable solution but the 
largest stable solution in this letter. The typical BP solution 
corresponds to the smaller stable solution because the 
initial value for the BP algorithm is commonly smaller than 
the unstable solution y u , e.g., see ll6lfT3l for CDMA systems. 
Thus, the inability of BP to get across the unstable solution 
makes the BP performance worse than the optimal perfor- 
mance when the potential energy U(y) is not monostable. 

In this letter, the potential energy U{y) is assumed to 
have multiple stable solutions. We hereafter refer to the 
largest stable solution, denoted by y + , and the other stable 
solution(s) as the optimal solution and the BP solution(s), 
respectively. 

3. Spatially-Coupled Dynamical System 

In order to obtain functionality of escaping from the BP so- 
lution^), we consider a collection of identical systems cou- 
pled in a one-dimensional manner. Let x e (-x max , x max ) 



y=yu 



s.y=y- 



y=y+ 



Fig. 1 Bistable potential energy U{y). 

denote the position of a system in the spatially coupling di- 
mension, with x max > defining the size of the whole sys- 
tem, and let y(x, t) be the state of the system at position x 
and at time t. The spatially-coupled system we consider is 
governed by the equation 



dy = _du (y) _ i D , IdyY + d_ ( D(y) dy\ 

dt dy 2 \dxj dx\ dxj' 



(4) 



where D(y) > is a positive coupling function. We study 
the system <j4j with the initial and boundary conditions 



y(x, 0) = yo e M for x e (-x 



max? -*max 



(5) 



y(±x maK , f) = y+. (6) 

Let (2N +1) denote the number of coupled systems. 
Threshold improvement occurs when the number of coupled 
systems, or equivalently N, tends to infinity and when the 
coupling strength tends to zero |@]|7]. The system © 

can be regarded as a space-continuum limit of (2N + 1) 
space-discrete coupled systems in jV — » oo, or an approx- 
imation of finite differences in the coupled systems. These 
interpretations are justifiable by letting A = x m!a /N and con- 
sidering (2N +1) copies of the original system at positions 
x = z*A for i = -N, -N+l, ...,N-l,N. As N gets large, the 
difference y((i+ 1)A, t)-y(iA, t) is expected to be sufficiently 
small, so that one can approximate the difference by 



dy A 2 d 2 y 

y((i + 1)A, t) - y(iA, t)^A-^- + —-^. 



(7) 



The difference (|7} is commonly included into a nonlinear 
function. Expanding the nonlinear function with respect to 
A yields a coupling function D(y) depending on the state 
i/(x, t). See Q for spatially-coupled CDMA systems. The 
last two terms on the right-hand side of © come from such 
approximations of space-discrete coupled systems. 

Threshold improvement for spatially-coupled CDMA 
systems can be understood from (|4} [7|. Unfortunately, the 
phenomenological model © does not include LDPC con- 
volutional codes. This is because a higher-order approxima- 
tion is needed near the boundaries for an ensemble of LDPC 
convolutional codes [4|, while the DE equation can be ap- 
proximated by (0]i in the bulk region far from the boundaries. 

The system © can be regarded as a dynamical sys- 
tem with the so-called Ginzburg-Landau free energy func- 
tional H(y) 1 14 1 as its potential energy 
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6H, 

Sy 



(y), 



with 



H(y) 



£ 



U (y(x, 0) + 1 g- 



(8) 



dx. (9) 



In ([8]), 5/5;/ denotes the functional derivative with respect to 
y. The energy functional (0 implies that we impose spatial 
coupling that smooths the state y(x, t) spatially. The point 
of spatial coupling is that the boundaries are fixed to the 
optimal solution y+. A "stretched rubber rope," both ends 
of which are fixed to the optimal solution y + , is utilized to 
"climb" the potential barriers between the BP solution(s) 
and the optimal solution. The tension of the rubber rope 
lifts the state y{x, t) toward the optimal solution. In LDPC 
convolutional codes, such a boundary condition results from 
termination of convolutional codes ||4) . 

4. Main Result 

For simplicity, we hereafter assume that the coupling func- 
tion does not depend on y, i.e., D(y) = D > 0. We believe 
that the main result holds for state-dependent coupling func- 
tions. We focus on stationary solutions y(x) to ©, satisfying 







dU 



d 2 y 



-^ ) + D df- 



y(±x m . dx ) = y+. 



(10) 



Figure |2] shows examples of the stationary solution y(x) for 
the double-well potential U(y) = y 4 /4 - y 2 /2 - hy with 
the parameter h e M. The double-well potential has a 
metastable solution t/_ < (resp. y+ > 0) and a stable 
solution y + > (resp. i/_ < 0) for h > (resp. h < 0). 
When h = 0.01, the state approaches the uniform solution 
y(x) = y+. When h = -0.01, on the other hand, the station- 
ary solution y{x) is a pot-shaped solution. Note that this so- 
lution is a natural solution for the case where the state y(x, t) 
cannot climb potential barriers. Pot-shaped stationary solu- 
tions also appear in LDPC convolutional codes. See |4| for 
the details. 

The main result of this letter is that there are no pot- 
shaped stationary solutions if the boundaries are fixed to the 
global stable solution. 

Definition 1: A stationary solution y(x) is called a pot- 
shaped solution if the following conditions are satisfied: 

• y(0) < j/(±x max ). 

• dy/dx > for x > 0. 

Note that any solution to (IToT t is an even function of x, 
because the differential equation (TT~0b is invariant under the 
transformation x' - -x. Thus, the second condition implies 
dy/dx < for x < 0. 

Theorem 1: Suppose that the coupling function D(y) > 
does not depend on y. If the largest stable solution y+ of 
U (y) is the unique global stable solution, then, there are no 
pot-shaped stationary solutions. 



uniform solution 



pot-shaped solution 



f!=-0.01 



-0.5 



Fig. 2 Stationary solutions to (|4]i for D = 0.01 and x max = 
1. The initial value yo is equal to 



Before proving Theorem Q] we shall present the sig- 
nificance of Theorem Q] Obviously, the uniform solution 
y(x) — y + is a stationary solution to the spatially-coupled 
dynamical system Intuitively, non-monotonic solutions 
on x e [0, x max ] (or x e [-x max ,0]) cannot become sta- 
tionary solutions to (|4]i with the uniform initial condition 
y(x, 0) = yo, because the state y(x, f) should move closer to 
y + as the position x gets closer to the boundary. Thus, The- 
orem[T]implies that the state y(x, t) converges to the uniform 
solution y(x) — y+ in t — > oo for any initial value, i.e., the 
state can approach the optimal solution for general potential 
energy U(y) such that the largest stable solution y + is the 
unique global stable solution. 

Remark 1: Suppose that the continuum approximation of 
the DE equation(s) for a spatially-coupled graphical model 
is given by ©. We assume that the potential energy U(y) 
is bistable and that it contains a parameter e, which corre- 
sponds to the erasure probability for LDPC codes over BEC 
or to the system load for CDMA systems. We rewrite the 
potential energy as f/(j/;e). The potential f/(j/_;e) at the 
smaller stable solution t/_ is assumed to be higher than that 
at the larger stable solution y+ for small e, as shown in Fig.Q] 
As e increases, the potential U(y-, e) at the smaller stable 
solution is assumed to get lower, while the potential at the 
larger stable solution gets higher. Eventually, the potential 
energy U(y; e) may have the same height at the two stable 
solutions at e = fg S p C) . Theorem Q] allows us to define the 
BP threshold for the spatially-coupled graphical model as 

.1. JSC) _ rr/_. . _(SCk rr/_. . iSCk 



the point eg S p C) 



at which U(y.; e^ S p C) ) =~ U(y + ; e^ s p C) ). 

JSC) 



BP threshold e^ p ' coincides with the optimal threshold for 
spatially-coupled CDMA systems [7|. It is unclear whether 

(SO 

the BP threshold e Bp coincides with the optimal one for any 
spatially-coupled graphical model. 

Proof of Theorem\J] We shall prove Theorem[T]by con- 
tradiction. Suppose that there is a pot-shaped stationary so- 
lution y(x). Integrating (TTOb after multiplying both sides by 
dy/dx, we obtain 
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pot-shaped solution 




uniform solution 



0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1 
D 

Fig. 3 Bifurcation diagram for x max = 1, obtained by nu- 
merical simulations of (0). The initial value yo is equal to 
the smaller stable solution. 



V 



dx 



(x)\ =U(y) + C, 



(ID 



with a constant C. We use the boundary condition 
i/(+x max ) = y+ and the positivity of the left-hand side of 
(TTTT > to find C > -U(y+), where we have used the assump- 
tion that y + is the unique minimizer of U(y). If y+ was a 
local minimizer, the inequality would have to be replaced 
by C > -U(y+) + AU, with AU > denoting the energy gap 
between y+ and the global minimizer. 

Since we have assumed that there is a stationary solu- 
tion satisfying dy/dx > (resp. dy/dx < 0) for x > (resp. 
x < 0), integrating (fTTT i after taking the square root of both 
sides yields 



F (y) = x — x for x > 0, 



with 



F(y) 




dy' 



^U(y') + C 



(12) 



(13) 



where y denotes a value between y(Q) and y+ that does not 
minimize the potential energy, i.e., U{y) > U{y+). In (fT2l . 
we have selected a constant of integration such that y(x) = y. 
Repeating the same argument for x < 0, we obtain 



y = F- 



— x) for x e (-x n 



(14) 



where F~ l denotes the inverse function of ([TBI . 

Any stationary solution must be differentiable since it 
is a solution to the second-order differential equation (11 Ot . 
However, the solution (fl4l is not differentiable at the origin 
unless dF~ l I dy\ y= y(0) — or dF / dy\ y=y (p) = °°, in which the 
value y(Q) at the origin is given by y(Q) - F~ l (-x). The 
uniqueness of the global stable solution implies that the in- 
tegrand in CfT~3T > can diverge only at y = y+. Thus, a necessary 
condition for dF/dy\y =y (0) = oo is y(0) = y + , which contra- 
dicts y(0) < (/(+x max ). □ 



What occurs when the boundaries are fixed to a 
metastable solution? Figure [3] shows a bifurcation dia- 
gram of stationary solutions for the double-well potential 
U(y) = j/ 4 /4 - y 2 /2 — hy with the parameter h. The larger 
stable solution y+ for h < corresponds to the metastable 
solution, i.e., the boundaries are fixed to the metastable solu- 
tion. The state converges to a pot-shaped stationary solution 
in t — > oo when (D, -h) is located above the curve shown in 
Fig. [3] Otherwise, the state converges to the uniform solu- 
tion y — y + . As long as D is finite, the state may be conveyed 
to the uniform solution y - y+ for small h. However, such 
a solution seems to disappear in D — > 0, which corresponds 
to the limit in which threshold improvement occurs |@]|2]- 
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